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Abstract 

This research aimed to evaluate of online tutorial program design by applying problem-based learning Research 
Methods currently implemented in the system of Open Distance Learning (ODL). The students must take a 
Research Methods course to prepare themselves for academic writing projects. Problem-based learning basically 
emphasizes the process of problem-solving, active learning, and collaborative learning in order to develop 
problem-solving skills and self-learning abilities. It is expected that the program will be implemented for the 
students as a learning program as eligible criteria for learning as well as for a good presentation in ODL systems. 
This research adopted the methods of Research-Based Development (Borg& Gall 2005), and several steps of 
research-based development methods were used to create effective problem-based learning for the Online 
Tutorial package of the Research Methods course. The researcher divided the research and development into five 
stages, namely, Need Analysis, Research Design, Materials Development, Lormative Evaluation Design 
Program, and Implementation Program. The population of study was all students in Semester 2015.2 who took 
the Research Methods course. The study sample consisted of 20 students, facilitated by two Online Tutor 
facilitators. Sample selection was carried out randomly regardless of ability or particular capacity. At this stage 
of development of the learning models, the evaluation model of learning was developed using the expert research 
method (expert judgments) and designers of learning. The tool is a measuring instrument used in the Evaluation 
Instrument of the Program. The finding of this research shows that Online Tutorial Program Design has already 
developed the criteria for learning as well as criteria for a good presentation of students’ Open Distance 
Learning. 

Keyword: Program Design, Online Tutorial, Problem-Based Learning, Research Based- Development, Research 
Methods, Open Distance Learning. 


A. INTRODUCTION 

Universitas Terbuka (UT) as one of the distance learning higher institution promotes independence in 
learning to its student. One of the efforts made by Universitas Terbuka (UT) that employs the Open Distance 
Learning (ODL) system in improving the quality of learning is to provide learning support services by 
implementing e-learning or Online Tutorials. Nowadays, most UT tutors implement e-learning; they have 
developed various types of e-learning materials. One of the learning support services that has been developed for 
online tutorials is widely known as e-learning. In running the ODL system, the use of e-learning can be classified 
into two parts: Lirst, the utilization of e-learning as the main instructional media. Second, the use of e-learning as 
instructional media support. This means that e-learning in a self-learning process plays a part in supporting 
learning media. E-learning as supporting instructional media should be available and ready to be used at any 
time by students and is regarded as compensation for face-to-face sessions. From 1996 up until now, UT has 
been consistently using e-learning, along with the development of the network technology era that has been 
growing very fast and is highly innovative. Advancement of e-learning has made it easier for developers to 
design learning support which is simple, quick and of high quality. E-learning has high capability in an 
interactive learning process. Therefore, continuous efforts need to be made to create a variety of patterns of 
thought and find the best alternative solutions to improve the quality of learning outcomes in the ODL system. In 
the era of network technology, improving the quality of students’ learning outcomes seems to be a very big 
challenge and has become a very important major issue, particularly for students who utilize the e-learning 
supporting services or online tutorials implemented by UT. 

The first step taken by the researcher was to identify the conditions and learning problems as well as the 
needs of those who take the Research Methods online tutorial and the learning outcomes. The course needs to 
provide supporting instructional media that offer learning support services for students. This research did not 
regard the learning support services and face-to-face tutorials as mandatory. Instead, the students can take online 
tutorials. Therefore, UT needs to provide an online tutorial package for the Research Methods course that does 
not regard learning support services and face-to-face tutorials as mandatory. Therefore, the researcher chose the 
Research Methods course for the development of a tutorial learning program in terms of design development. 
The purpose of the Research Methods course is to help students understand and have the ability to do research in 
the field. Therefore, the learning program is designed and developed based on problem-solving. There are still 
many questions to answer as follows. How is the online tutorial program implemented in the Research Methods 
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course which is currently part of the ODL system? Can the course help students understand and have the ability 
to do research. How should an online tutorial program based on problems in the Research Method course be 
developed in accordance with the learning criteria and the criteria for a good presentation in the ODL learning 
process system? The problems can be formulated as follows: “How can the design of an online tutorial program 
be developed based on problems in order to be implemented as an online tutorial program which is in line with 
the learning criteria and good presentation criteria in the ODL system.” From the above description, it can be 
concluded that in the learning process it is very important to use an online tutorial learning program which is 
effective and efficient based on problem-solving. As a result of this, the learning program serves as a source of 
information in delivering the learning materials, especially for students who have high potential. Finally, this 
research aimed to develop the design of an online tutorial learning program that meets the learning standard 
criteria and good presentation criteria, in accordance with the ODL learning process system. 

B. STUDY OF THE LITERATURE 

1. SELF-LEARNING PROCESS IN THE ODL SYSTEM 

Self-learning is a process of learning interactions between students with learning materials without the 
physical presence of the lecturer, either individually or in groups. Learning materials are designed in accordance 
with the principle of self-learning (self-instructional). The concept of self-learning is guided by philosophy 
which says that the learning process can occur without following a teaching process. According to Munir (2008), 
self-learning systems are based on the learners’ self-discipline and adjusted to the circumstances of individual 
learners including capability, speed of learning, willingness, interest, time, and socio-economic circumstances. In 
the ODL system, students conduct self-learning in the long term. The success of the learning process depends 
very much on being disciplined and time management. Students can have print and non-print media to study 
independently, prior to face-to-face tutorials or online tutorial sessions. In the era of self-learning information, 
learning can be done by using a computer connected to the Internet. One of the learning programs utilizing the 
Internet at UT is the online tutorial learning model. Therefore, online tutorial activities are expected to prepare 
students in order to be able to learn independently so that they can improve their learning outcomes and foster 
self-study in learning. In the learning process, students are given the opportunity to learn the materials and get 
feedback from the tutors through programs that are being used: Online tutorials, according to Heinich (1996), are 
technology-based learning media that serve as a means of interaction between tutors who act as resource persons 
and students as the receivers of the message, also referred to as e-learning. According to Kemp (1994), e- 
learning is the main choice in terms of an interactive learning process. The main characteristics of interactive 
learning are the students’ responses to the instructional materials and provision of feedback by the tutors 
regarding such responses. In the learning process, interaction takes place between the students and the tutors 
through a computer program. A well-designed computer program by the learning experts can create interactive 
communication between the students and the learning materials presented. In an interactive learning process, 
two-way-communication is very important. According Schwier (1993), the most important aspects in the 
interactive learning process is that the students are given the opportunities to interact. This can be done between 
students and students, between students and facilitators or tutors, and between students and e-learning materials. 
The use of e-learning that is carefully designed can improve the students’ activity in learning, as well as the 
students’ independence to initiate contacts, discussions, and reflection to improve their learning outcomes 

2. THE ROLE OF ONLINE TUTORIALS AS A LEARNING SUPPORT SERVICE 

The use of e-learning or online tutorials in education is very useful and effective to improve the learning 
process. The utilization of online tutorials in the learning process can provide advantages in terms of saving time 
and preserving a way of thinking that is more rational. Online tutorials are a learning support service for students 
which has the nature of academic activities. According to Afriani (2007), online tutorials are Internet-based 
tutorial services or web-based tutorials (WBT) which are offered by UT and attended by students through the 
Internet. In the ODL learning activities systems implemented by UT, online tutorial activities have a very 
important role because they can serve as a guidance of the students in the learning process. In online tutorials the 
students’ learning activities are conducted under the guidance of a tutor as facilitator, resource person, and 
coordinator of learning activities. Heinich (1996) said that an online tutorial begins with activities designed to 
lead students on the screen, so that students are ready to accept the lessons. As for the interaction between the 
tutors and the students associated with the information and knowledge given to the students, the tutors need to be 
communicative as if they are standing in front of the students and providing guidance directly to the students. 
The most common way to provide information is interactively by setting questions to be answered by the 
students. Questions should be repeated to attract the students’ attention, with exercises, and motivate students to 
have high motivation to learn. Having done so, the tutor will find out how well the students can remember and 
understand the information given. On the other hand, according to the tutor, a tutor plays both as a trigger and 
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motivator in the students’ learning process. The expected result is that the students will have the will and ability 
to observe, think, behave, and do when facing a concept of science and technology. According to Alessi & 
Trolip (1991), information and knowledge should be delivered in small units as the basic for a better 
understanding and developing e-learning well so that the students are motivated to learn accordingly. The 
contents delivery comprises questions or problems that require the students to give responses, response analysis, 
feedback, preparation practice until the students showed o the expected competencies. Meanwhile, according to 
Budianingsih (2007), e-learning is very instrumental in increasing the quality of learning as follows: 1. the e- 
learning for improvement and professional development of lectures; 2. as a source of study for learning; 3. as a 
learning interaction tool; and 4. e-learning as a place of learning, besides the learning paradigm change caused 
by the use of e-learning in study. E-learning has an important role in the professional development of lectures, 
tutors, and facilitators. Besides, e-learning can also be a communication tool and quick information provider 
between colleagues, lecturers, tutors, facilitators, and students. E-learning can be used to find the source of 
information the scope of which is very broad. E-learning is a very useful learning source for lecturers and 
students to follow the latest developments in the field of science. According to Budianingsih (2007), in the 
learning process, generally e-learning has the potential to empower students to encourage the growth of students’ 
learning skills, (learning to learn), students’ reasoning skills (higher-order thinking skills), communication skills 
(written or oral), and also the ability of students to find a variety of learning sources. The use of carefully 
designed e-learning can increase students’ activity in learning as well as the independence of students to initiate 
contact, discussion, and reflection to improve learning results. 

3. LEARNING APPROACH PROBLEM-BASED - LEARNING 

Learning-based problem-solving adapted from English Problem-Based Learning (PBL) is a learning 
approach that starts with solving a problem, but to solve the problem the students need new knowledge to 
complete it. Problem-based learning involves the students in an active learning process which is collaborative, 
student centered, and which develops the problem-solving and self-learning capabilities required to face the 
complex challenges today ( https://dinikomalasari.wordpress.com/20130) . An approach to learning involves 
students solving a problem through the stages of the scientific method so that students can learn the knowledge 
related to the problem and at once has skills to solve the problems. Learning approach problem solving in the 
learning process has a very important role. This learning approach is very useful to help students to solve various 
encountered problems. The learning problem-solving skills for the students are highly associated with increased 
learning outcomes. Therefore, in the learning process, online tutorials deliver research method material to the 
students and it is very necessary to use the problem solving approach. Problem-solving is a process of thinking 
and application of knowledge that has been gained before. Problem-solving concerns how to find a way out of a 
difficult situation full of obstacles in order to achieve the target. According to Gagne, learning by employing a 
problem-solving strategy is a way of complicated learning and the highest level compared with other types of 
learning. According to Polya, there is a five-step problem-solving approach that can be guidance for lecturers 
and students to implement problem-solving in the classroom. The lecturers’ task in this matter is: first, use 
guidance in directing students to solve the problem; second, enable the students to use this guidance when 
solving problems. The problem-solving guidance includes: 1) presenting the problem in a general form; 2) re¬ 
presenting the issues in the operational form; 3) determining strategies or procedures to resolve the problem; 4) 
solving problems; and 5) analyzing and evaluating the problem and the method of finding problem solving 
strategies. To solve a problem efficiently, the necessary sequence of events should be clear and systematic. 
Referring to some opinions above, researchers when designing learning programs should pay attention to three 
steps as follows: first step, present a problem. Second step, find out a variety of solutions based on the 
knowledge possessed. Third step, an evaluation of the solution is made. If the problem cannot be solved, then the 
problem-solving process can be started again from the first step. Therefore, the online tutorial learning program 
for the Research Method course that was developed was based on the contents of the learning materials of the 
Research Methods course. 

C. RESEARCH METHODS 

This research applied methods of Research and Development (R&D), often referred to as research- 
based development. Research-based Development is a process oriented toward development. Having modified 
the Research and Development (R&D) model, Borg and Gall (2005) classified the model into five stages. The 
first stage is Preliminary Study. The second stage is Design Research. The third stage is Development Program. 
The fourth stage is Trial Design Program (Formative Evaluation). The fifth stage is Implementation Program. 
The study population was 2015.2 semester students studying the course “Research Methodology” and the 
research sample for the tryout field consisted of 20 students, facilitated by two facilitators online. Selection of 
students was carried out randomly regardless of ability or capacity. In the learning model development stage, the 
targeting for the evaluation of the learning model in this case was a learning expert (expert judgment) such as: 
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author module Research Methods (Material Expert), Instructional Design Expert (Designer Instructional). The 
data collection in this research and development can be classified into four stages. Stage one: Preliminary Study, 
Second stage: Design Development Program, Third stage: Development Program and Fourth stage: Formative 
Evaluation. At each stage the selected data collection techniques accorded with their respective goals. In the 
Preliminary Study stage, selected data collection techniques were used together, such as questionnaire, 
observation, and documentation. Besides, a review of the literature was implemented. The data collection 
technique in the development stage consisted of trial via one-to-one evaluation and trial via small groups. An 
additional evaluation program conducted in this study used a modified Formative Evaluation Instrument of the 
Council of the Minister of Education Canada (CMC) and registration of documents. An instrument in the form of 
closed questions was aimed at students. Registration of documents was necessary to know the various documents 
used in the online tutorial activities. Interviews were conducted in the form of in-depth interviews to focus on 
finalizing the model. Data analyses and techniques at the early stage used descriptive analysis of research data 
through observation. The data analysis was conducted to determine the various requirements for the 
establishment of the program design. The instrument data was analyzed by grouping the data to determine the 
dominance of data on the items analyzed. 


Figure 1 : Steps of System Aproach Model of Educational Research and Development 



Source: Adapted from Figure 6 on pp. xxii-1 in: Dick, W., Carey, L., & Carey, J. O (2005). The systematic 
design of instruction (6th ed.). New York: Allyn & Bacon, Published by Allyn and Bacon, Boston, MA. 
Copyright 2005 by Pearson Education. Adapted with permission from the publisher. 

D. RESULTS AND DISCUSSION 

The purpose of this study was to implement the program design for online tutorials to meet the criteria for 
learning and good presentation criteria in the ODL system. The results of this study are then interpreted 
pertaining to the research objectives. In this research do development tutorial online design program-based 
problem-solving for the course research methods. According to Borg and Gall (1995) when implementing 
Research and Development (R&D) there are ten stages of research. Researchers have modified this study into 
five stages of research, so this study was simplified and the five stages of the research can represent the activities 
of the ten stages of the research. These stages are as follows: 

First Stage: Preliminary Study 

The Preliminary Study stage consists of a literature study, the results of previous studies, field studies and the 
analysis of the needs of the program design. Field studies are conducted using the method of observation and 
interview involving both the tutors and the students who took the research method course. The early researchers 
conducted a literature study/library by collecting supporting materials related to the concept of problem-solving 
methods and concepts based on development and implementation of the program design for online tutorials. 
Furthermore, researchers studied the results of previous studies. In general, the students are still experiencing 
problems in the technology related to online tutorial activity, especially in asking questions/comments on the 
forum, submitting assignments, and others. Ways of presenting this course, through online tutorials, do not 
facilitate students to understand the material. Initiation given by this program only resembles the summary 
module. At the early stage of lessons, the program does not provide the instructions for the students. The 
program does not provide additional training or links to other additional materials. Students are less active 
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following the discussion forums, so that this forum does not provide much benefit to the students. The results of 
field studies through interviews with the tutor illustrate the availability of: 1. lesson Plan tutorial online for all 
Tutor Research Methods; 2. implementation of the online learning process on a regular basis; 3. the existence of 
evaluation of learning outcomes by online tutorials; and 4. online tutorial tasks implemented regularly. The 
results of a field study on the analysis of the needs of the design aspects of the program through interviews of 
students illustrate that online tutorials are needed by students to improve their learning outcomes. Some of the 
students mentioned the difficulty in implementing the learning process through online tutorials. They have a 
wish that the developed program design be more simple and easy to follow. They are very happy to follow the 
learning process through online tutorials and very much needed, although some students are still experiencing 
some problems in accessing or opening the webpage of UT-Tutorial. 

Second Stage: 

The Second Stage is design program planning with the following procedures: early stage researchers Learning 
Objective Identification often called Instructional Goal for determining the ability or competence of students 
needed after taking this learning program. The formulation of the aim of study is produced through the process 
of needs analysis or need analysis and experiences of learning difficulties encountered by the students based on 
the results of preliminary studies. The formulated learning process also dealt with the analysis of specific tasks 
and the requirements to do the given tasks. Furthermore, after researchers have identified the learning objectives, 
then the next step is to develop: Instructional Analysis, which is a procedure used to determine the skills and 
knowledge that are relevant and needed by the students to achieve competence or their learning objectives. The 
instructional analysis process is easily done by using a map that illustrates the interconnections and relationships 
across skills and abilities required to achieve competence and has been determined as follows: a) formulating a 
general competence as the objective of the course concerned is the expected competence that can be achieved by 
the students after studying a certain course; b) dividing the general competency into several specific 
competencies required to achieve the general competence; and c) mapping or describing the relationship among 
those specific competencies to produce a map of competencies, and finally identifying initial students; behaviors, 
if any. 

Furthermore, researchers compiled Outline Online. Arranging Outline Online consists of: a) preparing the 
materials of research methods-based problem-solving (including the preparation of Initiations, Tasks, 
Discussions, and Exercises); b) producing a matrix of learning evaluation; and c) designing the instruments for 
evaluation the designed program. 

Stage Three: Development of Program Design 

At this stage of the design development program researchers adopt the steps of Dick & Carey’s model of 2005 
(Borg & Gall 2007). The development implemented relates to: a) initiation (online tutorial) of presentation 
material of Research Methods course-based problem-solving for 8 sessions; b) developing tasks; c) developing 
discussions; and d) developing exercises thus producing a tutorial online kit. Besides, researchers also developed 
a Learning Evaluation grille, as well as a Formative Evaluation (Evaluation Instrument) for the design of the 
research methods program. 

Fourth Stage: Trial & Revised Program Design (Formative Evaluation) 

Field tryout and revisions are a very important part of research and development. After the draft of the design 
development program is completed, the researchers run a trial, often termed as a formative evaluation. The 
purpose of this evaluation is to determine whether the design of the program developed is worth using or not. 
Can the program design meet the criteria of learning and presentation criteria? Formative evaluation or trial is 
implemented gradually as many as four times, namely: first, one-to-one evaluation by experts (expert test) and 
revisions. Expert tryout was conducted by the researchers involving one expert in online tutorial program design 
(learning technology specialist) , and content expert (methods expert) one person. During the tests, observation 
was made and interviews and questionnaires were conducted. The input of experts is to make a revision to 
improvements back to the design of the program that has been designed. Second, do test limited (one-to-one- 
evaluation) by learners for three students as users. Based on the findings of these trials, some revisions were 
made in order to improve the quality of online tutorial program design. After a limited trial test is done, then the 
third trial was done as a major one (small group evaluation) to small groups of 10 students. During the tests, 
observation was made and interviews and questionnaires were conducted. Based on the feedback coming from 
the students and instrument based on the distributed instrument, the next revision was conducted to further 
improve the designed the program. The fourth field tryout was carried out involving 20 students registered in 
semester 2 of 2015. Next revisions were made based on the input from the evaluation instruments given to 
students. Furthermore, after the program is revised, the program can considered as a final one, namely a model 
of online tutorial design program based-on problem solving used for the research methods course. 
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Figure 2. Formative Evaluation Procedures Design Program 
(Adapters Dick & Carey 2006). 


Based on the input data program, the program is already eligible for good learning based on the results as 
follows: 


No 

Statement 

Percent 

1 . 

stated that the introduction in the program motivates the students to learn the 
content of the program 

71% 

2. 

stated that the introduction of the program is helpful in preparing the 
students to learn the content of the program 

84% 

3. 

the students understood the purpose of the program 

79% 

4. 

the students answered that they did not find problems in accessing or 
opening the UT-online webpage 

83% 

5. 

the students did not experience technological constraints in doing the online 
tutorial activities, especially when asking questions / engaging in the forums 
of discussion, sending assignment 

75% 

6. 

the students stated that the way of presenting the material research methods 
facilitates students to understand the content of the material 

80% 

7. 

the students stated that they were very pleased to learn through the help of 
online tutorials 

90% 

8. 

the students stated that the online tutorial Initiation is like a summary of the 
modules that are considered difficult 

72% 

9. 

stated that the Initiation program, allows students to learn this material 

80% 

10. 

the students stated they actively opened Initiation tutorial online 

70% 

11. 

the students stated that the program provides many additional materials and 
additional links that help them understand the subject material 

73% 

12. 

the students stated that the program provides many additional exercises that 
help them understand the material subjects 

70% 

13. 

of the students stated that they had been active in the discussion forum 

71% 

14. 

of the students stated the discussion forum was very beneficial for their 
learning process 

75% 

15. 

of the students stated that they expressed their opinions or ask questions in 
the discussion forum 

70% 

16. 

expressed that in the feedback given in the forum of discussion was useful 
for them to learn more 

70% 

17. 

the students stated that the program content was beneficial for them 

75% 
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On the other hand, this program is already eligible criteria of a good presentation, based-on student assessment, 
__ namely: __ 


No. 

Statement 

Percent 

1 . 

the students stated the overview given at the initial program motivates them 
to learn the material presented in the program 

73% 

2. 

the students claimed that they understood the purpose of the program 

72% 

3. 

the students stated that the overview given in the program was useful in 
preparing them to learn the content of the program 

70% 

4. 

the students stated that the overview motivated them to learn the content of 
the program 

75% 

5. 

the students stated that the description of the program materials were 
broadcast interestingly 

76% 

6. 

the students stated that the delivery of materials helps them to understand the 
content of the initiation 

71% 

7. 

the illustration given helped the students understand the material content 

80% 

8. 

the students expressed that the initiation facilitates them to learn the material 
being presented 

75% 

9. 

the students said it was easy for them to go back and forth while operating 
the program 

70% 

10. 

the students stated that the instructions for using the program were quite 
clear 

74% 

11. 

the students stated that it ways easy for them to find information in the 
program 

70% 

12. 

the students stated the combination of colors used was quite good 

73% 

13. 

the texts in the programs were easy to read 

82% 

14. 

the students stated that animations exist within the program were in 
accordance with the content 

70% 

15. 

the students stated initiation presented in the program them understand the 
program content 

72% 

16. 

the students stated that the existing illustrations in the program helped them 
understand the program content 

75% 

17. 

The student stated that the illustrations were presented in congruous with the 
content 

80% 


Fifth Stage, Program Implementation 

The last stage is the stage of program implementation of online tutorial problem-solving based on the Research 
Methods course. This program can already be said to be final and already meets the criteria of learning and 
presentation criteria, and program implementation can be performed for other fellow students. 

E. CONCLUSION 

The results of this study show that the design of the online tutorial program involving the problem-based in the 
Research Methods course can be regarded as an eligible program in terms of the criteria which are in accordance 
with the defined research objectives. This program meets the requirements of the criteria in terms of learning 
since the students stated that among the 17 criteria, positive responses were (i.e., on average above 70%); while 
the 17 criteria have been highlighted in the results and discussion section. On the other hand, this program 
qualifies in terms of the criteria of presentation, because the students as the program users referred to the 17 
criteria associated the program; all those criteria received positive responses from the users (i.e., average above 
70%), while the criteria have been described in the results and discussion. In addition, the program has been 
design in accordance with the procedures and the development of theories that prevail in the development 
program used for online tutorials. 
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